{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "78dc1a9c-d009-407e-836d-84cf85936ade",
   "metadata": {
    "id": "78dc1a9c-d009-407e-836d-84cf85936ade"
   },
   "source": [
    "# AI Virtual Assistant for Customer Service\n",
    "\n",
    "## Content\n",
    "\n",
    "* [Overview](#Overview)\n",
    "* [Software Components](#Software-components)\n",
    "* [Key Functionality](#Key-functionality)\n",
    "* [How it Works](#How-it-works)\n",
    "* [Key Components](#Key-Components)\n",
    "* [Prerequisites](#Prerequisites)\n",
    "* [Deployment - hands on starts here](#Deployment)\n",
    "* [Getting API Keys](#Getting-API-keys)\n",
    "* [Docker Compose Check](#Docker-compose-check)\n",
    "* [Clone the Repository & Set Up Environment](#Clone-the-Repository-&-Set-Up-Environment)\n",
    "* [Build the Docker Containers](#Build-the-Docker-containers)\n",
    "* [Ingest Data](#Ingest-Data)\n",
    "* [Exposing the Interface for Testing](#Exposing-the-Interface-for-Testing)\n",
    "\n",
    "\n",
    "## Overview\n",
    "\n",
    "This Blueprint is showcasing an AI virtual assistant with NVIDIA NIM microservices (https://build.nvidia.com/nim)\n",
    "\n",
    "This blueprint is a reference solution for a text based virtual assistant. Companies are eager to enhance their customer service operations by integrating knowledge bases into AI assistants. Traditional approaches often fall short in delivering a combination of context-aware, secure, and real-time responses to complex customer queries. This results in longer resolution times, limited customer satisfaction, and potential data exposure risks. A centralized knowledge base that integrates seamlessly with internal applications and call center tools is vital to improving customer experience while ensuring data governance. The AI virtual assistant for customer service NVIDIA AI Blueprint, powered by NVIDIA NeMo Retriever™ and NVIDIA NIM™ microservices, along with retrieval-augmented generation (RAG), offers a streamlined solution for enhancing customer support. It implements context-aware, multi-turn conversations that feature general and personalized Q&A responses based on structured and unstructured data, such as order history and product details.\n",
    "\n",
    "This notebook will provide you with insights to the key components and walk you through its deployment and architecture in a step-by-step fashion. Note that this walk through is specific for the Docker Compose deployment. If you visit the [code repository](https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant), you will find additional information and other forms of deployment instructions (e.g. Helm chart deployment).\n",
    "\n",
    "## Software Components\n",
    "\n",
    "- NVIDIA NIM microservices\n",
    "    - Response Generation (Inference)\n",
    "        - NIM of meta/llama-3.1-70b-instruct\n",
    "        - NIM of nvidia/nv-embedqa-e5-v5\n",
    "        - NIM of nvidia/rerank-qa-mistral-4b\n",
    "    - Synthetic Data Generation for reference\n",
    "        - NIM of Nemotron4-340B\n",
    "- Orchestrator Agent - LangGraph based\n",
    "- Text Retrievers - LangChain\n",
    "- Structured Data (CSV) Ingestion - Postgres Database\n",
    "- Unstructured Data (PDF) Ingestion - Milvus Database (Vector GPU-optimized)\n",
    "\n",
    "Docker Compose scripts are provided which spin up the microservices on a single node. When ready for a larger-scale deployment, you can use the included Helm charts to spin up the necessary microservices. You will use sample Jupyter notebooks with the JupyterLab service to interact with the code directly.\n",
    "\n",
    "The Blueprint contains sample use-case data pertaining to retail product catalog and customer data with purchase history but Developers can build upon this blueprint, by customizing the RAG application to their specific use case. A sample customer service agent user interface and API-based analytic server for conversation summary and sentiment are also included.\n",
    "\n",
    "## Key Functionality\n",
    "\n",
    "- Personalized Responses: Handles structured and unstructured customer queries (e.g., order details, spending history).\n",
    "- Multi-Turn Dialogue: Offers context-aware, seamless interactions across multiple questions.\n",
    "- Custom Conversation Style: Adapts text responses to reflect corporate branding and tone.\n",
    "- Sentiment Analysis: Analyzes real-time customer interactions to gauge sentiment and adjust responses.\n",
    "- Multi-Session Support: Allows for multiple user sessions with conversation history and summaries.\n",
    "- Data Privacy: Integrates with on-premises or cloud-hosted knowledge bases to protect sensitive data.\n",
    "\n",
    "By integrating NVIDIA NIM and RAG, the system empowers developers to build customer support solutions that can provide faster and more accurate support while maintaining data privacy.\n",
    "\n",
    "## How it works\n",
    "\n",
    "This blueprint uses a combination of retrieval-augmented generation and large language models to deliver an intelligent, context-aware virtual assistant for customer service. It connects to both structured data (like customer profiles and order histories) and unstructured data (like product manuals, FAQs) so that it can find and present relevant information in real time.\n",
    "\n",
    "The process works as follows:\n",
    "\n",
    "- User Query: The customer asks a question in natural language.\n",
    "- Data Retrieval: The system retrieves relevant data—such as support documents or order details—by embedding and searching through internal databases, product manuals, and FAQs.\n",
    "- Contextual Reasoning: A large language model uses these retrieved details to generate a helpful, coherent, and contextually appropriate response.\n",
    "- Additional Capabilities: Tools like sentiment analysis gauge the user’s satisfaction and conversation summaries help supervisors quickly review interactions.\n",
    "- Continuous Improvement: Feedback from interactions is fed back into the system, refining the model’s accuracy and efficiency over time. The end result is a virtual assistant that can understand complex questions, find the right information, and provide personalized, human-like responses.\n",
    "\n",
    "### Key Components\n",
    "\n",
    "The detailed architecture consists of the following components:\n",
    "\n",
    "**Sample Data** The blueprint comes with synthetic sample data representing a typical customer service function, including customer profiles, order histories (structured data), and technical product manuals (unstructured data). A notebook is provided to guide users on how to ingest both structured and unstructured data efficiently.\n",
    "\n",
    "Structured Data: Includes customer profiles and order history Unstructured Data: Ingests product manuals, product catalogs, and FAQs\n",
    "\n",
    "**AI Agent** This reference solution implements three sub-agents using the open-source LangGraph framework. These sub-agents address common customer service tasks for the included sample dataset. They rely on the Llama 3.1 model 70B and NVIDIA NIM microservices for generating responses, converting natural language into SQL queries, and assessing the sentiment of the conversation.\n",
    "\n",
    "**Structured Data Retriever** Works in tandem with a Postgres database and Vanna.AI to fetch relevant data based on user queries.\n",
    "\n",
    "**Unstructured Data Retriever** Processes unstructured data (e.g., PDFs, FAQs) by chunking it, creating embeddings using the NeMo Retriever embedding NIM, and storing it in Milvus for fast retrieval.\n",
    "\n",
    "**Analytics and Admin Operations** To support operational requirements, the blueprint includes reference code for managing key administrative tasks:\n",
    "\n",
    "- Storing conversation histories\n",
    "- Generating conversation summaries\n",
    "- Conducting sentiment analysis on customer interactions These features ensure that customer service teams can efficiently monitor and evaluate interactions for quality and performance.\n",
    "\n",
    "**Data Flywheel** The blueprint includes a robust set of APIs, some of which are explicitly designed for feedback collection (identified by 'feedback' in their URLs). These APIs support the process of gathering data for continuous model improvement, forming a feedback loop or 'data flywheel.' While this process enables refinement of the model's performance over time to improve accuracy and cost-effectiveness, it is important to note that they do not directly perform the model fine-tuning itself.\n",
    "\n",
    "**Summary** In summary, this NVIDIA AI Blueprint offers a comprehensive solution for building intelligent, generative AI-powered virtual assistants for customer service, leveraging structured and unstructured data to deliver personalized and efficient support. It includes all necessary tools and guidance to deploy, monitor, and continually improve the solution in real-world environments.\n",
    "\n",
    "![Blueprint Diagram](https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant/raw/main/docs/imgs/IVA-blueprint-diagram-r5.png)\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "### Docker compose\n",
    "\n",
    "#### System requirements\n",
    "\n",
    "Ubuntu 20.04 or 22.04 based machine, with sudo privileges\n",
    "\n",
    "Install software requirements\n",
    "- Install Docker Engine and Docker Compose. Refer to the instructions for Ubuntu.\n",
    "- Ensure the Docker Compose plugin version is 2.29.1 or higher.\n",
    "- Run docker compose version to confirm.\n",
    "- Refer to Install the Compose plugin in the Docker documentation for more information.\n",
    "- To configure Docker for GPU-accelerated containers, install the NVIDIA Container Toolkit.\n",
    "- Install git\n",
    "\n",
    "By default the provided configurations use GPU optimized databases such as Milvus.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35890619-980d-4176-bbcb-c696a411d83f",
   "metadata": {
    "id": "35890619-980d-4176-bbcb-c696a411d83f"
   },
   "source": [
    "# Deployment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caf74f11-ef47-44b5-8f75-fc9e2a6fcf55",
   "metadata": {},
   "source": [
    "## Install required modules\n",
    "\n",
    "Restart the kernel after running following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52809252-4116-48dd-affd-5567f3e35e42",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install openai"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e7b9159-d45b-4f4b-99f8-d857c360dd86",
   "metadata": {},
   "source": [
    "## Getting API Keys - Very Important\n",
    "\n",
    "To run the pipeline you need to obtain an API key from NVIDIA. These will be needed in a later step to Set up the environment file.\n",
    "\n",
    "- Required API Keys: These APIs are required by the pipeline to execute LLM queries.\n",
    "\n",
    "- NVIDIA API Catalog\n",
    "  1. Navigate to **[NVIDIA API Catalog](https://build.nvidia.com/explore/discover)**.\n",
    "  2. Select any model, such as llama-3.3-70b-instruct.\n",
    "  3. On the right panel above the sample code snippet, click on \"Get API Key\". This will prompt you to log in if you have not already.\n",
    "\n",
    "NOTE: The API key starts with nvapi- and ends with a 32-character string. You can also generate an API key from the user settings page in NGC (https://ngc.nvidia.com/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e293133",
   "metadata": {},
   "source": [
    "Export API Keys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "044596c7-801c-4bfa-b9f6-e940cab81993",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "044596c7-801c-4bfa-b9f6-e940cab81993",
    "outputId": "8f4c4700-9cc2-47f4-e122-3b1ae38b8ab3"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "NVIDIA_API_KEY = input(\"Please enter your NVIDIA API key (nvapi-): \")\n",
    "NGC_API_KEY=NVIDIA_API_KEY\n",
    "os.environ[\"NVIDIA_API_KEY\"] = NVIDIA_API_KEY\n",
    "os.environ[\"NGC_CLI_API_KEY\"] = NGC_API_KEY\n",
    "os.environ[\"NGC_API_KEY\"] = NGC_API_KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7L1yGbPtGF4q",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 680
    },
    "id": "7L1yGbPtGF4q",
    "outputId": "c11b4632-d2c2-44ba-b373-a9d767a317cd"
   },
   "outputs": [],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(\n",
    "  base_url = \"https://integrate.api.nvidia.com/v1\",\n",
    "  api_key = f\"{NVIDIA_API_KEY}\"\n",
    ")\n",
    "\n",
    "completion = client.chat.completions.create(\n",
    "  model=\"meta/llama-3.1-70b-instruct\",\n",
    "  messages=[{\"role\":\"user\",\"content\":\"Write a limerick about the wonders of GPU computing.\"}],\n",
    "  temperature=0.2,\n",
    "  top_p=0.7,\n",
    "  max_tokens=1024,\n",
    "  stream=True\n",
    ")\n",
    "\n",
    "for chunk in completion:\n",
    "  if chunk.choices[0].delta.content is not None:\n",
    "    print(chunk.choices[0].delta.content, end=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fba880dd",
   "metadata": {
    "id": "fba880dd"
   },
   "source": [
    "## Docker Compose check\n",
    "Ensure the Docker Compose plugin version is 2.29.1 or higher."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b222afe-67e0-4a7d-b450-dff89e9bc22b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7b222afe-67e0-4a7d-b450-dff89e9bc22b",
    "outputId": "017c4b6a-f1bf-44e0-aac4-82e49d3f1d85"
   },
   "outputs": [],
   "source": [
    "# Check certain versions and packages installed\n",
    "!docker compose version"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d29236d",
   "metadata": {
    "id": "3d29236d"
   },
   "source": [
    "## Clone the Repository & Set Up Environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe83be0d-7c6a-493d-aa85-ac2e2cd168ae",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fe83be0d-7c6a-493d-aa85-ac2e2cd168ae",
    "outputId": "88cd1b34-d3c8-48f2-9b34-1c52ec95594f"
   },
   "outputs": [],
   "source": [
    "#  Clone the Repository\n",
    "!git clone https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26290dfc",
   "metadata": {
    "id": "26290dfc"
   },
   "source": [
    "The purpose of this code snippet below is to ensure that the notebook is operating within a directory named \"ai-virtual-assistant\". If it's not, it changes to that directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f400b89-f9ca-41b4-95f0-eb771c88c617",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "4f400b89-f9ca-41b4-95f0-eb771c88c617",
    "outputId": "2a8b0af1-e666-44dc-c1d3-a8440ae825c2"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "current_path = os.getcwd()\n",
    "last_part = os.path.basename(current_path)\n",
    "\n",
    "if os.path.basename(os.getcwd()) != \"ai-virtual-assistant\":\n",
    "    os.chdir(\"ai-virtual-assistant\")\n",
    "\n",
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02e1eeec",
   "metadata": {
    "id": "02e1eeec"
   },
   "source": [
    "We login into the NGC catalogue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ded8d982",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 106
    },
    "id": "ded8d982",
    "outputId": "14a0124b-1ae5-4136-b4f6-8d0ddb7b9843"
   },
   "outputs": [],
   "source": [
    "!docker login nvcr.io -u '$oauthtoken' -p $NGC_API_KEY"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdd03972",
   "metadata": {
    "id": "cdd03972"
   },
   "source": [
    "## Build the Docker containers\n",
    "\n",
    "We are launching the containers by using the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7387b26",
   "metadata": {
    "id": "a7387b26"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "docker compose -f deploy/compose/docker-compose.yaml up -d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b618cbbc-4e6c-44db-85d0-b5dbe4617f33",
   "metadata": {
    "id": "b618cbbc-4e6c-44db-85d0-b5dbe4617f33"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "## Ensure the containers are spun up and look healthy\n",
    "docker ps --format \"table {{.ID}}\\t{{.Names}}\\t{{.Status}}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dd0234b",
   "metadata": {
    "id": "0dd0234b"
   },
   "source": [
    "## Download data\n",
    "\n",
    "Download the manuals into data/manuals_pdf folder Run this script to download the manuals listed in the specified txt file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d28623b0-d7c9-4502-9f38-0a41e0f4ba35",
   "metadata": {
    "id": "d28623b0-d7c9-4502-9f38-0a41e0f4ba35"
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "# Ingest data - download data\n",
    "./data/download.sh ./data/list_manuals.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a935841d-e2ba-4509-bc33-c24d26b31f83",
   "metadata": {},
   "source": [
    "## Ingest data\n",
    "\n",
    "Open the jupyter notebook  \"./ai-virtual-assistant/notebooks/ingest_data.ipynb\" and run through the cells (Shift + Enter) to ingest the structured and unstructured data types.\n",
    "**NOTE:** The first cell in the ingest_data.ipynb requires you to input the proper IP address of the localhost. If the machine is spun up with a default container on Brev, this ought to be the default Docker IP: 172.17.0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74eeec5a",
   "metadata": {
    "id": "74eeec5a"
   },
   "source": [
    "## Exposing the Interface for Testing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f76c683",
   "metadata": {
    "id": "4f76c683"
   },
   "source": [
    "The Blueprint comes equiped with a basic UI for testing the deployment. This interface is served at port 3001. In order to expose the port and try out the interaction, you need to follow the steps below.\n",
    "\n",
    "First, navigate back to the created Launchable instance page and click on the Access menu.\n",
    "\n",
    "\n",
    "![Access Menu](https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant/raw/main/docs/imgs/brev-cli-install.png)\n",
    "\n",
    "\n",
    "Scroll down until you find \"Using Tunnels\" section and click on Share a Service button.\n",
    "\n",
    "\n",
    "![Using Tunnels](https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant/raw/main/docs/imgs/brev-tunnels.png)\n",
    "\n",
    "\n",
    "Enter the port 3001, as that is where the UI service endpoint is. Confirm with Done. Then click on Edit Access and make the port public:\n",
    "\n",
    "\n",
    "![Share Access](https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant/raw/main/docs/imgs/brev-share-access.png)\n",
    "\n",
    "\n",
    "Past this point, by clicking on the link, the UI should appear in your browser and you are free to interact with the assistant and to ask him about the data that was ingested.\n",
    "\n",
    "\n",
    "![AI Virtual Assistant Interface](https://github.com/NVIDIA-AI-Blueprints/ai-virtual-assistant/raw/main/docs/imgs/ai-virtual-assistant-interface.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "815b4220",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
